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ABSTRACT 

It  is  important  in  the  age  of  the  INTERNET  and  video  games  to 
keep  students  engaged  in  the  classroom.  One  way  of  doing  this  is 

TM 

through  the  use  of  games  and  simulations.  TABOO  is  a  card  game 
which  involves  getting  players  to  say  a  keyword  without  saying  five 
forbidden  words1.  This  can  be  adapted  for  any  number  of  different 
subjects.  In  this  paper  the  game  was  adapted  for  use  in  “Detection 
and  Estimation”  (EENG663)  class  at  the  Air  Force  Institute  of  Tech¬ 
nology  (AFIT).  A  study  was  conducted  to  see  if  there  is  a  difference 
in  the  scores  of  the  final  examination  for  two  different  groups.  Data 
was  obtained  on  the  students.  Their  project,  midterm  and  final  exam 
scores  were  recorded.  Group  one  took  EENG663  without  the  addi¬ 
tional  game  preparation,  group  two  had  the  additional  review  given 
right  before  the  final  exam.  A  statistical  analysis  of  the  scores  is 
presented  in  this  paper.  Some  students  also  completed  a  survey  that 
asked  their  opinions  on  using  the  game  in  the  classroom.  Results 
show  that  there  was  a  statistically  significant  difference  in  the  mean 
final  exam  scores  for  the  two  groups. 

Index  Terms —  games,  education,  review,  graduate,  analysis 

1.  INTRODUCTION 

There  has  a  been  a  push  in  the  past  few  years  to  increase  the  num¬ 
ber  of  students  entering  and  retained  in  the  field  of  science,  technol¬ 
ogy,  engineering  and  math  (STEM).  In  a  STEM  retention  study  of 
selective  universities  (ACT  score  21-27)  only  33%  of  students  grad¬ 
uated  after  five  years  LI]-  This  has  prompted  research  into  ways  to 
improve  the  number  of  students  that  actually  graduate  with  STEM 
degrees  [2],  [3],  [4],  [5],  According  to  Singer  [6J,  activities  that  ac¬ 
tively  engage  students  are  activities  where  professors  replace  part  of 
the  lecture  with  activities  that  invite  student  participation.  The  ref¬ 
erence  highlights  the  importance  of  diversifying  teaching  practices 
in  order  to  continually  engage  students  in  STEM  fields  and  suggests 
that  overall  evidence  supporting  active  learning  is  strong.  Pedagog¬ 
ical  advantages  to  using  games  in  the  classroom  has  been  well  doc¬ 
umented  throughout  the  years  and  their  use  is  supported  by  a  range 
of  learning  theories  [7],  [8]. 

The  use  of  games,  simulations  and  multi-media  projects  is  a 
form  of  active  learning.  Active  learning,  in  which  the  student  is  an 
active  participant  in  their  learning  experience,  leads  to  the  student 
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being  engaged  in  the  classroom.  Alternatively,  passive  learning  is 
a  more  traditional  or  lecture-based  way  of  teaching  and  it  is  more 
difficult  to  keep  students  engaged  [9],  A  good  example  of  imple¬ 
menting  games  in  the  classroom  may  be  found  in  Paulson  et  al.  [9], 
“For  example,  when  students  are  introduced  to  the  concepts  of  “laws 
of  nature”  and  “the  scientific  method,”  it  is  hard  to  convey  through 
lectures  the  nature  of  scientific  work  and  the  fallibility  of  inductive 
hypotheses.  Instead,  students  play  a  couple  rounds  of  the  Induction 
Game,  in  which  playing  cards  are  turned  up  and  either  added  to  a 
running  series  or  discarded  according  to  the  dealers  pre-conceived 
“law  of  nature”.  Students  are  asked  to  “discover”  the  natural  law  by 
formulating  and  testing  hypotheses  as  the  game  proceeds.” 

A  literature  review  on  the  use  of  games  in  graduate  level  classes 
produced  results  for  business,  education,  English,  science  and  man¬ 
agement  [10],  [9],  [11],  [12],  [3].  However,  a  further  refinement 
of  this  search  to  include  engineering  graduate  level  courses  did  not 
provide  any  helpful  results.  Since  the  use  of  a  board  game  to  review 
for  an  exam  in  a  graduate  level  signal  processing  class  is  not  well 
documented,  this  paper  will  attempt  to  fill  in  the  gaps. 

We  will  analyze  the  effects  (if  any)  from  supplementing  a  tradi¬ 
tional  review  for  a  final  examination  with  an  adaptation  of  the  game 

TM 

TABOO  ,  herein  called  DETABOO.  Classes  that  participated  in 
EENG663  for  the  past  two  years  at  the  Air  Force  Institute  of  Tech¬ 
nology  (AFIT)  were  chosen  for  this  analysis.  The  first  class  (group 
1)  did  not  participate  in  the  supplemental  DETABOO  review,  while 
the  second  class  (group  2)  did.  The  authors  are  interested  in  identify¬ 
ing  any  similarities  or  differences  in  the  scores  of  exams  and  projects 
for  these  two  years.  Additionally,  a  survey  was  given  to  several  stu¬ 
dents  in  the  class  which  participated  in  DETABOO  to  gauge  their 
opinions  on  playing  DETABOO  in  the  classroom. 

2.  DETABOO 

The  game  took  place  during  a  typical  class  period  of  approximately 
fifty  minutes.  122  DETABOO  cards  (see  example;  Figure  1)  are  di¬ 
vided  equally  among  two  teams  of  about  ten  students,  but  conditions 
may  vary  as  long  as  the  ratios  stay  the  same.  Additionally,  a  buzzer, 

TM 

card  holder  and  hour  glass  timer  (from  TABOO  game  set)  were 
used.  The  rules  for  DETABOO  are  shown  below. 

1.  Divide  class  into  two  teams,  each  team  choses  a  Clue  Giver 
(CG).  Cards  are  placed  in  a  blind  so  that  only  the  CGs  may 
see  each  others  cards  and  their  teams  may  not.  Each  student 
rotates  and  takes  a  turn  as  CG.  Each  CG  should  be  able  to 
get  through  about  five  cards  before  the  sand  falls  through  the 
hourglass. 
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Fig.  1.  Example  of  DETABOO  cards 


2.  Team  A’s  CG  starts  with  a  stack  of  cards  and  Team  B’s  CG 
sits  beside  Team  A’s  CG.  Team  A’s  CG  draws  a  card.  The 
word  at  the  top  of  the  card  is  the  “Guess  Word”  that  Team  A’s 
CG  is  trying  to  get  their  team  to  say.  The  five  words  below 
this  are  words  that  the  CG  may  not  use  to  describe  the  word. 

3.  Team  B’s  CG  sounds  a  buzzer  if  Team  A’s  CG  says  one  of 
the  “Forbidden  Words”  or  breaks  a  rule.  Rules  include;  no 
gestures,  noises,  “sounds  like”,  initials,  abbreviations  and  no 
using  parts  of  the  “Forbidden  Words”  or  “Guess  Word”.  If 
a  CG  is  “buzzed”  their  team  will  lose  a  point  .  Players  may 
pass  on  cards,  but  this  will  also  lose  them  a  point. 

4.  Each  time  the  team  shouts  out  the  correct  guess  word  they  are 
rewarded  with  a  point,  this  goes  until  the  timer  runs  out.  Once 
the  time  has  elapsed  the  CGs  switch  roles  and  Team  B’s  CG 
proceeds  with  clues  to  their  team.  The  teams  rotate  like  this 
until  teams  run  out  of  cards,  the  team  with  the  most  points 
wins.  Each  student  has  a  chance  to  be  a  CG. 
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The  basic  TABOO  cards  were  adapted  for  use  in  the  EENG663 
class  and  some  example  cards  are  shown  in  Figure  1 . 

3.  SURVEY 

A  survey  was  given  to  group  two  to  assess  their  opinions  on  partic¬ 
ipating  in  DETABOO  as  a  supplemental  review  for  the  final  exami¬ 
nation.  Students  were  asked  to  rank  statements  regarding  the  use  of 
DETABOO  in  the  classroom.  Answers  were  scored  based  on  a  Lik¬ 
ert  scale  from  one  to  five,  with  one  representing  “strongly  disagree”, 
three  representing  “neutral”  and  five  representing  “strongly  agree”. 
The  five  questions  listed  in  Table  1  all  begin  with:  “I  feel  that”. 

Due  to  the  nature  of  the  classes  at  AFIT,  it  was  not  possible  to 
survey  all  students  as  several  had  already  left.  Every  attempt  was 
made  to  obtain  as  many  surveys  as  possible,  but  our  sample  size  for 
the  surveys  is  only  seven.  Since  no  actual  analysis  is  being  done  with 
these  results  sample  size  should  not  be  a  concern.  This  survey  was  to 
gauge  students  opinions  on  using  DETABOO.  Ideally  future  surveys 
would  be  given  to  the  students  before  they  left  the  class.  Summary 
results  of  the  survey  are  shown  in  Table  1.  Each  question  is  listed 
on  the  side  and  above  three  categories,  NEUTRAL,  AGREE  and 
DISAGREE  are  displayed.  Here  we  combine  “mostly  agree”  with 
“agree”  and  “mostly  disagree”  with  “disagree”,  since  they  both  are 
similar  responses.  The  percentage  of  respondents  in  each  category 
is  listed  in  the  cells.  Rows  sum  to  100%. 

The  majority  of  students  felt  that  DETABOO  was  a  good  supple¬ 
ment  to  the  traditional  lecture  style  review,  with  only  one  student  re¬ 
porting  negatively.  This  is  interesting  since  the  majority  of  students 
did  not  feel  that  playing  the  game  helped  their  performance.  No  stu¬ 
dents  felt  that  the  game  hurt  their  performance  on  the  final  exam. 


Table  1.  Survey  Results 


QUESTION 

DISAGREE 

NEUTRAL 

AGREE 

1 -DETABOO  was  a 
good  compliment  to  the 
review. 

14% 

14% 

72% 

2-Time  would  have 
been  better  spent  with 
more  traditional  review. 

14% 

43% 

43% 

3-DETABOO  helped 
my  performance  on  the 
final  exam. 

86% 

14% 

0% 

4-DETABOO  hurt  my 
performance  on  the  fi¬ 
nal  exam. 

71% 

29% 

0% 

5-Professor  should  use 
this  in  the  future. 

14% 

43% 

43% 

When  asked  if  the  professor  should  continue  to  use  DETABOO  in 
future  classes  the  majority  of  students  reported  positively.  In  general 
it  seems  that  students  responded  positively  to  using  DETABOO  as 
a  supplement  to  the  traditional  final  exam  review.  No  student  felt 
that  they  were  harmed  by  playing  DETABOO.  It  seems  that  students 
welcome  a  break  from  the  traditional  review,  even  if  it  is  of  no  direct 
benefit  to  them.  Obviously  the  implications  of  this  study  are  limited 
given  the  small  sample  size.  However  since  negative  impact  appears 
to  be  minimal  it  would  seem  that  no  harm  would  be  felt  by  students 
if  the  professor  continued  to  use  DETABOO  in  the  classroom. 


4.  DESCRIPTION  OF  DATA  AVAILABLE  FOR  GROUPS 

The  subjects  of  interest  here  are  two  groups  of  students  at  AFIT; 
students  who  participated  in  DETABOO  and  students  that  did  not 
participate.  These  graduate  students  took  one  of  two  classes.  Group 
one  attended  class  during  the  winter  term  2009  and  did  not  partici¬ 
pate  in  DETABOO  as  a  supplemental  final  exam  prep.  Group  two 
attended  class  during  Winter  Term  of  2010  and  did  participate  in  the 
additional  DETABOO  review,  before  their  final  exam.  Thus  for  this 
study  we  will  assume  both  test  groups  are  homogeneous,  except  with 
regards  to  their  DETABOO  experience.  All  exams  and  projects  were 
of  equal  difficulty,  although  they  were  modified  slightly  to  avoid  stu¬ 
dents  knowing  the  answers  to  questions  from  previous  years  exams 
and  project.  Due  to  the  nature  of  classes  at  AFIT  we  will  assume 
all  of  our  subjects  are  drawn  from  the  same  sample  population.  The 
demographics  of  the  sample  population  are  shown  below. 

1 .  90%  of  group  are  military  officers 

2.  95%  of  the  group  are  male 

3.  Range  in  age  from  20-40 

4.  Majority  have  Bachelors  and /  or  Masters  in  Engineering 

5.  All  were  Masters  or  Doctoral  students  in  engineering  fields 


5.  STATISTICAL  ANALYSIS 

We  will  attempt  to  answer  several  different  questions.  Our  primary 
research  question  is:  What  are  the  effects  (if  any)  on  adding  a  sup¬ 
plemental  final  exam  prep  in  the  form  of  DETABOO? 
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Table  2.  Descriptive  Statistics-Final  Exam 


Group 

n 

Mean 

Std.  Dev 

Min 

Max 

No  DETABOO 

18 

67.94% 

15.98% 

42% 

92% 

DETABOO 

18 

77.67% 

15.91% 

44% 

94% 

5.1.  Initial  Data  Analysis 

An  initial  examination  of  the  final  exam  scores  will  be  helpful  to  see 
if  our  research  question  is  justified.  Initial  data  analysis  identified 
two  possible  outliers  in  our  data  set.  These  outliers  contained  the 
most  extreme  studentized  residuals,  see  Kutner  et  al.  [13]  for  fur¬ 
ther  explanation  on  this  procedure.  Table  2  provides  the  descriptive 
statistics.  The  data  show  that  the  mean  final  exam  score  is  higher  for 
those  students  that  participated  in  DETABOO  than  for  those  that  did 
not.  Additionally,  both  the  minimum  and  maximum  test  scores  for 
group  two  are  larger  than  those  values  for  group  one.  It  is  reasonable 
to  assume  that  there  is  a  difference  in  the  mean  final  exam  scores  for 
the  two  groups. 

5.2.  T-test 

A  T-test  is  used  to  look  for  differences  in  the  mean  for  the  midterm, 
project  and  final  exam  score  for  each  group.  Refer  to  [14]  for  a 
general  description  of  T-tests.  An  important  step  prior  to  perform¬ 
ing  the  T-test  is  to  establish  our  significance  level  (a)  and  calculate 
the  degrees  of  freedom.  The  criterion  used  for  rejecting  the  null  hy¬ 
pothesis,  a,  is  defined  as  the  probability  of  a  Type  I  error  or  “false 
positive”.  A  Type  1  error  would  occur  if  we  concluded  that  the  mean 
scores  differ  when  in  fact  they  actually  do  not. 

a  =P(Type  I  error)  =  P(Reject  iTo|iTo  is  true)  (1) 

d.f  =  m  +  ri2  —  2  (2) 

The  choice  of  a  level  of  significance  is  not  based  on  any  mathemat¬ 
ical,  statistical  or  substantive  theory.  It  is  a  choice  which  is  purely 
arbitrary  for  our  research  since  we  can  not  attach  a  cost  to  either  a 
Type  I  or  Type  II  error  [15],  Thus  we  will  set  our  significance  level 
to  a  =  0.10.  The  probability  of  a  Type  II  error,  /?,  is  generally  not 
defined  but  the  reader  may  refer  to  [16]  for  further  explanation.  A 
Type  II  error  occurs  if  we  concluded  that  the  mean  scores  did  not 
differ  when  they  actually  do  differ.  The  actual  amount  of  variability 
in  the  sampling  distribution  of  T  depends  on  the  sample  size.  This 
dependence  is  expressed  by  degrees  of  freedom  (d.f).  For  each  pa¬ 
rameter  we  are  estimating  we  lose  a  d.f.  Thus  if  we  are  estimating 
two  means  we  would  lose  two  degrees  of  freedom. 

Having  established  the  initial  parameters,  we  may  proceed  with 
calculations  of  the  test  statistic,  T,  and  p-value.  The  T-statistic  is 
the  ratio  of  how  much  our  data  mean  scores  differ  from  each  other 
by  their  total  standard  error.  This  is  compared  to  a  critical  value, 
To,  and  a  decision  on  whether  to  reject  the  null  hypothesis  is  made 
if  the  T  —  stat  falls  outside  of  the  rejection  region,  defined  on  the 
right  hand  side  of  Equation  (4).  To  may  be  found  by  looking  at  a 
standard  T-table  available  in  many  statistics  texts.  The  p-value  or 
observed  significance  level  is  the  probability  of  observing  a  value 
of  the  test  statistic  that  is  at  least  as  extreme  the  test  statistic  that 
was  calculated  from  our  data,  assuming  the  null  hypothesis  is  true 
[16]  .  When  the  null  hypothesis  is  rejected  the  result  is  said  to  be 
statistically  significant.  The  T-test  statistic  and  p-value  are  calculated 
using  Equations  (3)-(4)  respectively. 


Table  3.  T-test  results  for  differences  in  all  scores  for  both  groups 


Null  Hypothesis 

T-stat 

d.f 

p-value 

Ho  :  pp2  =  PPi 

0.73 

34 

0.47 

Ho  :  pmt2  =  Pmti 

-0.78 

34 

0.45 

Hq  '■  Pfe2  =  pFEi 

1.8289 

34 

0.04 

S1  |  s2 
n\  '  n2 

p  —  value  =  P{\T\  >  To)  (4) 

where  xi  is  the  mean  exam  score  of  group  i,  (i  e  { 1, 2}),  si  is  the 
variance  of  data  from  group  i ,  and  rn  is  the  sample  size  of  group  i, 
with  m  =  ri2  =  19.  With  34  degrees  of  freedom,  and  a  =  0.10, 
we  have  To  =  1.306.  Our  rejection  region  is,  T  <  —1.306  or 
T  >  1.306.  For  any  value  that  falls  outside  of  this  region  we  would 
reject  the  null  hypothesis  and  conclude  that  there  is  a  difference  in 
the  mean  scores  for  the  two  groups. 

Table  3  gives  the  corresponding  null  hypotheses,  T  —  statfd.f) 
and  p— value  for  project  (pp),  midterm  ( pmt )  and  final  exam (pfe) 
scores  respectively.  The  alternative  hypothesis  for  the  final  exam 
scores  is:  Hi  :  p2  >  pi,  that  the  students  in  the  group  two  scored 
better  on  the  final  exam  than  those  group  two.  For  final  exam  we 
choose  a  one-sided  hypothesis  since  we  are  interested  in  showing 
that  scores  for  group  two  are  greater  than  group  one.  For  both  the 
midterm  and  project  scores  our  alternative  hypotheses  is  that  there 
is  no  difference  in  the  mean  scores,  Hi  :  p2  ^  pi,  here  a  two-sided 
hypothesis  is  used  because  we  are  interested  in  any  difference  in  the 
two  groups. 

For  project  and  midterm  scores  we  fail  to  reject  the  hypotheses 
at  an  a  =  0.10  level.  There  is  insufficient  evidence  to  conclude 
that  there  is  a  difference  in  mean  scores  of  the  two  groups  for  the 
project  and  midterm  scores.  The  T  test  for  the  final  exam  score  has  a 
p—  val  =  0.04.  Thus,  at  an  ct  =  0.10  level  we  would  reject  the  null 
hypothesis  and  conclude  that  the  mean  final  exam  scores  for  those 
students  who  had  DETABOO  was  greater  than  for  those  that  did  not 
have  DETABOO. 

In  order  to  quantify  the  difference  between  the  mean  test  scores, 
we  may  look  at  the  quantile  values  for  both  groups  [14].  Figure  2 
is  a  graphical  representation  of  the  quantile  final  exam  values  for 
each  group.  For  every  quantile,  students  in  the  additional  review 
group  scored  higher  on  the  final  exam  than  those  students  without 
the  review.  This  means  that  if  a  student  scored  in  the  50tfe  quantile 
on  the  final  exam;  they  received  a  72%  on  their  final  if  they  did  not 
play  DETABOO  but  an  84%  if  they  did  play.  It  should  be  noted  that 
we  did  not  show  quantile  calculations  on  the  other  variables  because 
their  corresponding  T-test  were  not  statistically  significant. 

From  the  two  previous  results  we  can  conclude  that  for  our  two 
groups,  there  is  a  difference  in  mean  final  exam  scores  and  those  that 
had  DETABOO  appear  to  score  better  on  the  final  exam.. 

5.3.  Regression 

Another  way  to  examine  the  effects  of  participating  in  DETABOO  is 
through  the  use  of  a  simple  linear  regression.  Regression  is  the  study 
of  dependence.  Linear  regression  is  the  most  commonly  used  type  of 
regression  and  implies  a  linear  relationship  between  the  independent 
and  dependent  variables  [13], 
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Fig.  2.  Quantile  Plots  of  Final  Exam  Scores  for  both  Groups 


Table  4.  Regression  Results 


Coefficient 

Value 

Std.  Error 

T-Stat 

p-value 

67.94 

3.76 

18.08 

<  0.0001 

Pi 

9.72 

5.31 

1.83 

0.08 

Equation  (5)  gives  the  initial  model  that  we  wish  to  examine. 
With  Yi  being  our  final  exam  score  (our  output)  and  X.t  is  a  dummy 
variable  representing  whether  or  not  the  student  played  the  game  (in¬ 
put).  The  regression  coefficients  ( f3o ,  /3i)  were  calculated  by  MAT- 
LAB,  refer  to  Weisberg’s  text  [8]  for  equations. 


Yi=p0  +  fiiXi 


(5) 


where. 


f  1,  DETABOO 
\  0,  NO  DETABOO 


(6) 


Here,  /3o  represents  the  constant  coefficient,  /3i  is  the  coefficient  as¬ 
sociated  with  the  indicator  variable.  Table  4  gives  the  results  of  a 
regression  analysis  for  Equation  (5).  The  fit  for  our  model  would  be, 


Yi  =  67.94  +  9.72  (X;).  (7) 


At  an  a  =  0.10  level  our  /%  coefficient  is  significant.  We  may 
conclude  that  those  students  who  had  DETABOO  review  on  average, 
increased  their  final  exam  score  by  approximately  9.72%. 


6.  CONCLUSION  AND  FUTURE  WORK 

Through  our  statistical  analysis  we  have  shown  a  significant  differ¬ 
ence  in  the  final  exam  scores  for  students  in  the  two  sample  groups. 
Further,  it  was  shown  that  on  average  students  that  had  DETABOO 
scored  approximately  ten  points  better  on  their  final  exam  then  those 
students  that  did  not  participate.  It  was  also  shown  ther  there  was  no 
discernible  difference  in  students  midterm  and  project  scores.  This 
shows  it  is  reasonable  to  assume  that  both  groups  of  students  per¬ 
formed  equally  in  all  aspects  of  the  class  except  for  the  final  exam 
scores  where  those  with  DETABOO  performed  better.  With  such  a 
small  sample  size  we  are  cautious  in  our  interpretations.  However, 
since  the  implications  of  this  study  will  not  have  detrimental  effects 
to  the  students  the  authors  feel  that  this  analysis  serves  its  purposes. 
Since  much  research  exists  tauting  the  benefits  of  active-learning  in 
the  classroom  [6],  the  authors  feel  that  implementing  this  type  of  re¬ 
view  would  have  a  positive  effect  on  students.  This  agrees  with  our 
survey  results.  Students  respond  positively  to  playing  DETABOO 


and  recommended  that  the  professor  continue  to  use  it,  even  though 
they  believed  that  playing  DETABOO  gave  them  no  advantage  on 
the  final.  This  might  suggest  that  students  welcome  a  change  to  the 
traditional  type  of  engineering  graduate  level  classes.  The  authors 
would  like  to  expand  the  scope  of  the  study  to  include  classes  the 
professor  will  teach  during  the  2011-2012  school  year.  This  would 
provide  the  authors  with  more  test  data  as  well  as  more  survey  results 
that  would  be  beneficial  in  future  analysis. 
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